The field of the invention is searching, and in particular searching for information stored in a set of websites.
A website (xe2x80x9csitexe2x80x9d) is defined herein as a collection of files stored on a computer (e.g., a server) that is connected to a network. The World Wide Web (WWW) is a collection of websites whose servers are interconnected through the Internet. A collection of websites can also be stored on servers that are interconnected through a private network, e.g., through an intranet.
In many cases, at least some of the files of a website contain hyperlinks. A hyperlink is typically a text, graphic or image object in a first file that, when selected by a user, either causes a second file to be displayed to the user, causes a different part of the first file to be displayed to the user, or executes a program. In this way, a file in a website can be interrelated with another file stored at the same website, a different website, or elsewhere. The interrelated files of a single website usually reflect a common theme, such as information about a particular company, activity, or service.
The amount of information stored in a collection of websites can be substantial. For example, the WWW includes over 600,000 websites. Conservatively assuming an average data size of 2 Megabytes (MB) per website, the WWW includes over 1200 billion bytes of information across a wide range of topics. Finding a particular piece of information in such a large collection can be problematic. For example, simple browsing through the websites in search of a particular type of information can be impractical in a website collection of substantial size.
One known system addresses the problem of finding particular information stored at websites by categorizing websites according to the topic or topics to which they pertain. One such known system is the Yahoo! search engine located at  less than http:  www.yahoo.com greater than . Yahoo! obtains information about the topic or theme to which a website pertains along with a brief narrative describing the contents of the website (i.e., from the administrator or owner of the website). This information (along with a website identifier) is then correlated with a category. The Yahoo! categories are organized hierarchically, so that a given category typically has one or more subcategories, and each such subcategory has further subcategories, etc.
An example of a Yahoo! interface is shown in FIG. 1. An example of a category is ArtsandHumanities, 101, which has subcategories Literature 102 and Photography 103. When a user selects the Literature subcategory 102, Yahoo! displays the page shown in FIG. 2 to the user. FIG. 2 shows numerous subcategories 201 of the Literature subcategory 102. Hereinafter, the term xe2x80x9ccategoryxe2x80x9d will be used interchangeably with the term xe2x80x9csubcategory.xe2x80x9d
Yahoo! also accommodates keyword searching. In FIG. 2, a user has entered a search for the keyword xe2x80x9ctelephonexe2x80x9d 202 that is restricted 203 to websites in the Literature category. In this case, the user may be interested in finding literature where the telephone plays a major role. When the search button 204 is selected, only website descriptions, and not website content, that fall under the category xe2x80x9cLiteraturexe2x80x9d are searched for the term xe2x80x9ctelephone.xe2x80x9d Website descriptions are generally terse, one line or one paragraph summaries describing the content of the website. A website description cannot fully capture all of the detail contained in the website""s content. Indeed, by definition, it is a summary. Because only the descriptions are keyword searched, and not the content, a Yahoo! keyword search can disadvantageously miss relevant content even when the keyword search is limited to website descriptions in a relevant category. Websites whose descriptions contain the term xe2x80x9ctelephonexe2x80x9d are displayed to the user, as shown in FIG. 3.
As discussed above, because Yahoo! keyword searches only search the descriptions of websites and not their content, a Yahoo! keyword search can miss identifying websites that contain information relevant to the user""s request. Thus, for example, many files at different websites in the Literature category may well contain the keyword xe2x80x9ctelephone.xe2x80x9d None of these would be detected and displayed to the user by Yahoo!, even though the user is interested in finding occurrences of xe2x80x9ctelephonexe2x80x9d in websites that fall within the Literature category. In this way, the Yahoo!-type category/descriptive information search is overly narrow, and is prone to miss detecting information that the user would be interested in seeing.
Another known system for searching for information at websites stores and indexes a vast amount of content from numerous websites, but does not correlate website content with categories. Such a known system is the AltaVista(trademark), located at  less than http://www.altavista.digital.com greater than . In AltaVista(trademark), a user submits a keyword search. FIG. 4 shows the AltaVista(trademark) interface in which a user has submitted a keyword search request for the term xe2x80x9cATandTxe2x80x9d 401. In response, AltaVista(trademark) searches its stored content for occurrences of the term xe2x80x9cATandTxe2x80x9d, and shows the user the websites that have content in which the term occurs (402.) Some excerpted content (e.g., 403) is also displayed. It is difficult for the user to efficiently and accurately identify websites that have content of interest to the user.
Just as the Yahoo!-type search can be too narrow, the AltaVista(trademark)-type content search can be too broad. For example, the results for the keyword search shown in FIG. 4 include over 300,000 websites 404. Even when the results are organized in some prioritized fashion (e.g., websites with the greatest number of occurrences of the keyword term are listed first), such a broad result is too large to be very useful to the user.
Searching by category and then using a keyword search to search the descriptive information about websites within a category can be too narrow, and miss detecting websites that have content that is relevant to the user""s request. On the other hand, keyword searching of only the content of websites can be too broad. A way is needed to take advantage of the narrowing effect of a category search and the depth of a content search to yield a more accurate and complete search result.
In accordance with an embodiment of the present invention, websites are searched for desired information first by narrowing the scope of the search by identifying websites that correspond with a category pertinent to the desired information. Next, a keyword search is carried out on the content (not just the descriptions or summaries of content) of websites that fall within the pertinent category. This is advantageously more efficient than searching all of the content of the universe of websites initially, because such a search often disadvantageously returns too many results, many of which can be irrelevant (e.g., as in Altavista(trademark)) Likewise, it provides higher resolution than simply performing a category search, which can fail to identify websites within the category that have the most relevant information. It also provides higher resolution than narrowing the field of websites by category, and then performing a keyword search on website descriptions or content summaries, e.g., as in Yahoo!, which can miss relevant information that is included in the content itself, but not in the description or summary. The present invention advantageously combines the efficiency and accuracy of category and content searching to provide a more efficient, better way of finding the information most relevant to a user""s need in a set of websites.